Skip to content

Conversation

BenjaminBraunDev
Copy link
Contributor

@BenjaminBraunDev BenjaminBraunDev commented Sep 27, 2025

/kind bug
/kind feature

What this PR does / why we need it:
PostResponse runs at the first chunk received instead of at the end of the request, however some PostResponse plugins would like to run only when a request is fully complete. This PR allows that flexibility by breaking out PostResponse into 3 separate requestcontrol plugin hooks for when the response is received, each chunk streamed, and fully complete respectively. The requestcontrol plugins are now the following:

PreRequest
PostResponseReceived
PostResponseStreamed
PostResponseComplete

Which issue(s) this PR fixes:
Fixes #1483

Does this PR introduce a user-facing change?:

PostResponse plugin has been renamed to PostResponseReceived

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. kind/feature Categorizes issue or PR as related to a new feature. labels Sep 27, 2025
Copy link

netlify bot commented Sep 27, 2025

Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 646e952
🔍 Latest deploy log https://app.netlify.com/projects/gateway-api-inference-extension/deploys/68ed63ba4159180008585688
😎 Deploy Preview https://deploy-preview-1661--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 27, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @BenjaminBraunDev. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 27, 2025
@ahg-g ahg-g removed the kind/bug Categorizes issue or PR as related to a bug. label Sep 28, 2025
@ahg-g
Copy link
Contributor

ahg-g commented Sep 28, 2025

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 28, 2025
@kfswain
Copy link
Collaborator

kfswain commented Oct 3, 2025

Some smaller comments, overall looks good. Thanks!

/approve

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 3, 2025
}

// HandleResponseBodyStreaming is called every time a chunk of the response body is received.
func (d *Director) HandleResponseBodyStreaming(ctx context.Context, reqCtx *handlers.RequestContext, logger logr.Logger) (*handlers.RequestContext, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove logger from the args, we don't use logr as argument to functions, we use contextual logging.
for example

log.FromContext(ctx).V(logutil.TRACE).Info(....)

see for example here:

loggerTrace := log.FromContext(ctx).V(logutil.TRACE)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I asked him to remove that, in the streaming case the high volume of instantiations of the logger will cause allocation/gc pressure. Luke mentioned this as an optimization during flow control benchmarking, and i think the same applies here b/c there will be multiple streaming calls per request.

Copy link
Contributor Author

@BenjaminBraunDev BenjaminBraunDev Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does log.FromContext(ctx).V(logutil.TRACE).Info(....) actually make another logger? I would think it takes the logger "from the context" given ctx has a logger. Does it actually allocate more memory? I would think that pretty inefficient.

From the FromContext() in go-logr package:

// FromContext returns a Logger from ctx or an error if no Logger is found.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nirrozenbaum @kfswain What is the desired state for this? I can revert it back or keep the changes

Copy link
Contributor

@nirrozenbaum nirrozenbaum Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it makes another logger.
that wouldn't make sense.

the desired state (at least in my opinion) is to remove logger from the args and use contextual logging if/when necessary.
we don't pass around logger in any part of the code. we pass ctx (which includes inside it the logger).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, keeping logger in parameters. I don't have a strong opinion on this beyond we probably shouldn't create a logger for every chunk streamed.

Copy link
Contributor

@nirrozenbaum nirrozenbaum Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a confusion here.
@kfswain the call Logger.WithValues creates a new instance of the logger (with some key value pairs attached to it). we create a logger only ONCE somewhere up in the handling chain and store it inside the context using log.IntoContext(ctx, logger). for example here:

ctx = log.IntoContext(ctx, logger)

then, in all other functions, we call log.FromContext(ctx). This call does NOT create a new logger. it retrieves the stored logger from the context. more information here:
https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/log#pkg-functions

The best practice in contextual logging is to create logger once and store it into the context, and retrieve it from context in the various places.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah sorry, mixed messaging here, the initial implementation had the WithValues.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm slightly confused here, the FromContext in log.go always calls log.WithValues(keysAndValues...), it's the only way the function can return. Should I keep this logging regardless?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was checking now again. I see your point.
I’m still not sure about that, but anyway this looks to me like a system wide issue cause we’re using contextual logging in all places.

I suggest to move forward with current PR using Log.FromContext, open a dedicated issue to check about this issue, and if we decide to change, we need to update everywhere, not only in this function.

let me know if it makes sense.

@nirrozenbaum
Copy link
Contributor

@BenjaminBraunDev made another iteration on the PR.
overall it looks good. the main point I was highlighting in the review is mainly around keeping consistency and use contextual logging.
otherwise the PR lgtm.

@kfswain
Copy link
Collaborator

kfswain commented Oct 6, 2025

/approve

Thanks Ben! Will let Nir give final stamp

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenjaminBraunDev, kfswain

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unexpected behavior of PostResponse plugins for streaming requests

5 participants